Parallel computing systems are generally made up of many nodes, each of which is capable of performing data computation independently of the other nodes. Applications written to exploit this parallelism distribute their workload across multiple nodes in distinct processes. In such situations, nodes generally need to communicate with other nodes to share data. To achieve this sharing of data, a communication protocol may be used.
MPI, or Message Passing Interface, is a type language-independent communication protocol used to program parallel computers. MPI is not sanctioned by any major standards body; nevertheless, it has become the defacto standard for communication among processes that model a parallel program running on a distributed memory system. Actual distributed memory supercomputers such as computer clusters often run these programs. MPI is a specification, not an implementation. MPI has Language Independent Specifications (LIS) for the function calls and language bindings. The implementation language for MPI is different in general from the language or languages it seeks to support at runtime. Most MPI implementations are done in a combination of C, C++ and assembly language, and target C, C++, and Fortran programmers. However, the implementation language and the end-user language are in principle always decoupled.
One challenge of using a communication protocol such as MPI is tuning certain parameters to optimize performance of the communication protocol for a given application. Testing and selecting these parameters manually often requires hours to weeks of tedious tuning work and must be repeated for each unique application to be run on a parallel computing system. Similarly, any change to the composition of the parallel computing system (e.g. number of nodes, change in communication fabric, etc.) requires a new set of parameters to be tuned for optimal performance. Automation of such a process may speed up the process considerably, saving many hours of tedious work.